Census sample: the sampling rule as one .dag fold, checked against the pin at the pin's revision - #12475

Census sample: the sampling rule as one .dag fold, checked against the pin at the pin's revision

Follow-up to #12334, agreed with the XL-2 manager (option A of the escalation). reference_conservation_stratified_sample_paths was a hand-copied list with the selection rule stated only in prose. DESIGN §6 says "name the instrument"; this gives the list an instrument.

What is added (v2.compiler.reference_conservation_census)

• sample_paths_by_rule(files), the rule's single authority.
  • It keeps .dag files under dag/ and src/v2/ and sorts them by path. One pass then groups the strata: every stratum's paths share a prefix ending at a segment boundary, so sorting makes each stratum a contiguous run.
  • Strata are the first 2 path segments under dag/ and the first 3 under src/v2/.
  • Quota: k = max(6, round_half_even(300 * stratum count / total)), capped at the stratum's small-file count.
  • Selection: indices floor(i*n/k) over the stratum's files of at most 8,000 bytes, sorted by path.
• sample_file_within_cap(content). UTF-8 never takes fewer bytes than code points, so text over 8,000 code points is over the cap without being encoded. Only shorter text is encoded to count its bytes.
• reference_conservation_stratified_sample_revision records the pin's revision next to the list (8fcd8e77b8…), so the pairing is explicit.
• reference_conservation_sample_by_rule_at_pin_exit, the instrument. It reads the tree at the pinned revision from the git object store (gunbc.namespace_step0_subject_collector collect_step0_subject_vector_at), applies the rule, and exits non-zero, naming both differences, unless the result is exactly the pinned list.

Why at the pin's revision and not the checked-out tree. The sample is a snapshot. Comparing it to a tree that has since grown would red on every PR that adds or resizes a .dag file: a change detector, not a check (DESIGN §5). Checked at its own revision, a difference can only mean the pin or the rule is wrong.

Receipt (the instrument, run once)

gunbc run --source-root dag --source-root src/v2 \
  --entry src/v2/compiler/reference_conservation_census.dag \
  --function reference_conservation_sample_by_rule_at_pin_exit
→ rc=0 (ExitSuccess) in 536 s; read all 6,880 .dag blobs at 8fcd8e77b8 (== git ls-tree count)

The rule, in .dag, reproduces the pinned 316-path list exactly.

Recorded, not hidden:

• The original pin at fec339d561 was produced by a Python one-off that took indices with a float step. At that tree, the float step and exact floor(i*n/k) differ in one stratum.
• The current pin agrees with the exact form. The exact form is now the rule, and the list's comment says so.

Controls (floor, supplied file sets; each red by mutation)

In v2.test.claim.namespace_xl0.reference_conservation_census:

All claims are green locally. The three large fixture file sets are nullary and enrolled warm in v2.workflow.floor_pure_producer_share, so each claim pays only for the rule it runs.

🤖 Generated with Claude Code